Spiking Neural-Networks-Based Data-Driven Control

Machine learning can be effectively applied in control loops to make optimal decisions robustly. There is increasing interest using spiking neural networks (SNNs) as the apparatus for machine engineering because SNNs potentially offer high energy efficiency, and new SNN-enabling neuromorphic hardware being rapidly developed. A defining characteristic of problems that environmental reactions delayed rewards must considered. Although reinforcement (RL) provides fundamental mechanisms address such problems, implementing these SNN has been underexplored. Previously, spike-timing-dependent plasticity schemes (STDP) modulated by factors temporal difference (TD-STDP) or reward (R-STDP) have proposed RL with SNN. Here, we designed implemented an controller explore compare two considering cart-pole balancing a representative example. TD-based rules are very general, resulting model exhibits rather slow convergence, producing noisy imperfect results even after prolonged training. We show integrating understanding dynamics environment into function R-STDP, robust SNN-based learned much more efficiently than TD-STDP.